Language and Speech Processing

At the end of the 19th century, L. L. Zamenhof proposed Esperanto; it was intended as a global language to be spoken and understood by everyone. The inventor was hoping that a common language could resolve global problems that lead to con ict. This idealistic idea did not reach its full potential, yet there are still scienti c elds pursuing its legacy, though not necessarily from an ideological point of view. The Internet contains billions of web pages, as they come in all kinds of languages, a great deal of information is not available to us. A practical application would be a browser that translates these pages in a preferred language. In the eld of statistical machine translation (SMT), we try to build algorithms to translate from one language to the other by mere statistics taken from large bi-text corpora. When we adopt the SMT approach, we represent all individual or groups of words (cepts) as having a connection to zero, one or many foreign cepts under a probability value. This means that both the alignments and the probabilities need to be extracted from the bi-text. The main problem here is that we need the alignments to estimate the probabilities and the probabilities to estimate the alignments. These kinds of problems can be solved with the EM algorithm, one approach [2, 3] is particularly favored due to its estimation of reasonable models. In this paper, we will present the SMT Aligner (SMTA), which word aligns French-English sentences given word translation probabilities provided by [1] to estimate good alignments under the assumptions of IBM model I. We will then compare results from alignments that use the null word with ones that don't. We will also introduce a heuristic to increase the F1-score results by 4 percent. In section 2 we describe the theory which forms the basis of the SMTA. Section 3 is used to describe the research methodology. In Section 4 we present our results and we conclude section 5 with the discussion.